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(54) Image decoding with a simpiified process 

(57) A decoding unit (12) decodes an image accord- 
ing to JPEG 2000. A simplifying unit (30) comprises an 
automatic transformer (32) which compares an elapsed 
time to a time limit at each stage of decoding the image 
and switches the decoding process to a simplified proc- 



ess if necessary. For instance, if the playback of a mo- 
tion picture takes over 1/30 second, the simplified proc- 
ess, in which only low frequency components are de- 
coded, is conducted by the automatic transfonner (32). 
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Description 

[0001] This invention relates to an image decoding 
technique. This invention especially relates to a method 
and apparatus for decoding coded image data. 5 
[0002] In recent years digital image culture has be- 
come commonplace in our daily life as various informa- 
tion devices such as personal computers, digital cam- 
eras and color printers have been introduced into our 
homes and the number of Internet users has exploded. 
Image compression technology such as JPEG (Joint 
Photographic Expert Group) and MPEG (Motion Picture 
Expert Group) for still images and motion images re* 
spectively has been standardized. Image distribution 
and reproduction have become easy and convenient for 
users using recording media such as CD-ROM and 
transmission media such as networks and broadcasting 
technology based on the aforementioned image com- 
pression technology. JPEG 2000 has been announced 
in the JPEG line of technology. In MPEG also, future 
target specifications have been planned and discussed. 
There is no doubt that image processing technology will 
soon play an important role in improving the functionality 
of digital devices. 

[0003] For instance, recent digital cameras give users 
access to a convenient function of shooting of motion 
pictures, and are therefore taking over the original prov- 
ince of video cameras. The number of pixels in CCD or 
charge-coupled devices is of an order of millions and 
some cameras have the function of a high-speed con- 
tinuous shoot. Although some users stick to cameras 
using film, no one can resist the tide of the digitization 
which is rising as music records are replaced by com- 
pact discs. 

[0004] However, in order that these devices can be 
further developed as commercial products, functionality, 
operability and cost must be considered together. The 
enhancement of functionality demands high operability 
and low cost. For instance, if a function for the playback 
of a motion picture or a function for high-speed contin- 
uous shooting is provided with a digital camera, the im- 
age data compressed and stored into a memory card 
and so forth must be retrieved, decoded and enlarged 
in time for display. For this purpose, a high-speed image 
decoder must be incorporated into the digital camera 
and therefore it is likely to be disadvantageous In cost, 
power consumption and in the size of the implementa- 
tion. 

[0005] The present invention has been made with a 
view to the above-mentioned problems, and an object 
thereof is to provide an image decoding technique that 
is advantageous in cost and power consumption. 
[0006] Japanese Patent Application Laid-open No. 
H1 1-225337 discloses a technology in which a sender 
of image data breaks off data transmission when appro- 
priate, according to the dispersion of the speed of the 
transmission path. It is also disclosed therein that a 
wavelet transfomn, to which the present invention can 



be applied, is utilized. This technology is mainly de- 
signed not to work when the transmission path is in a 
nomial state. On the other hand, the present invention 
is independent of the transmission path and aims at cost 
reduction and so forth on the decoding side. 
[0007] The objects are achieved by combinations de- 
scribed in the independent claims. The dependent 
claims define further advantageous and exemplary 
combinations of the present invention. 
[0008] According to one embodiment, a method for 
decoding a coded image is provided. The method com- 
prises switching an ensuing part of a decoding process 
to a simplified process whenever the coded image is not 
completely decoded within a predetermined time. 
Therefore, without a complicated process or a high- 
speed process, a certain level of image quality can be 
maintained and an undesirable playback with frame 
drop can be prevented. The situation in which the de- 
coding of the coded image is not completed. Implies a 
condition in which, judged from its rate of progress, it is 
highly probable that the decoding of the image will not 
be completed within an appropriate time, as well as the 
situation in which the decoding of the image is not ac- 
tually completed. 

[0009] This embodiment may comprise obtaining the 

coded image, subjecting the obtained coded image to a 
decoding process, monitoring an elapsed time of the de- 
coding process, and switching an ensuing part of a de- 
coding process into a simplified process when it is de- 
tected that the elapsed time at a checkpoint in the de- 
coding process exceeds a predetermined time limit, 
[0010] According to another embodiment, a method 
for decoding a coded image is provided. The coded im- 
age is a motion picture coded on a frame-by-frame ba- 
sis, and the method comprises switching an ensuing 
part of a decoding process of any frame included in the 
coded image to a simplified process whenever the frame 
is not completely decoded within a predetermined time. 
The predetermined time may be, for instance, 1/30 sec- 
ond, that Is, 30 frames are decoded and played as a 
motion picture within one second, or 1/5 second, that is, 
5 frames are decoded and played as a high-speed con- 
tinuous picture. 

[001 1 ] The decoding process of the frame may be di- 
vided into a plurality of steps and a time limit may be 
defined for each of the steps, and when an elapsed time 
reaches the defined time limit at any one of the steps, 
the ensuing part of the process of the step may be sim- 
plified. The plurality of the steps may comprise the steps 
for decoding each of the color components or the steps 
for decoding each of the image regions, for instance, 
however a given step may be any stage or phase In 
which the time limit and the elapsed time can be com- 
pared. 

[0012] The coded Image may be a motion picture in 
which the frames are coded independently of each other 
and the simplified process may be a process confined 
within each of the frames. In this case, since the simpli- 
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tied process can be conducted on each frame separate- 
ly, the result obtained does not affect the Image quality 
of another frame. 

[0013] According to still another embodiment, an im- 
age decoding apparatus is provided. The apparatus 
comprises a decoding unit which decodes a coded im- 
age, and a simplifying unit which switches an ensuing 
part of a decoding process to a simplified process in 
such a situation that the coded image is not completely 
decoded within a predetemiined time. The coded image 
may be a motion picture coded on a frame basis and in 
this case the simplifying unit may switch the decoding 
process to the simplified process on the frame basis. 
The simplifying unit may substitute for the decoding unit 
by conducting a simplified decoding process by itself, or 
may simply direct the decoding unit to conduct a simpli- 
fied process. 

[0014] The simplifying unit may compare a time limit 
defined for each of a plurality of frame-decoding steps 
to an elapsed time at each of the steps, and simplify an 
ensuing part of the process of any one of the steps dur- 
ing which the elapsed time reaches the time limit. The 
simplifying unit may conduct the comparison of the time 
limit and the elapsed time and initiate the simplified proc- 
ess if necessary at each of the steps of decoding each 
of a plurality of color components which compose the 
image or decoding each of a plurality of regions which 
compose the image. The simplified process is not nec- 
essary as long as the time limit is not exceeded. How- 
ever, even in such a case, the simplified process may 
be conducted for another reason. 
[0015] The simplifying unit may relax the time limit al- 
located to a remaining step when any one of the steps 
is completed earlier than or within its time limit. Macro- 
scopically, the simplifying unit may relax the predeter- 
mined time allocated to the next frame to be decoded 
when the decoding process of any one of the frames is 
completed earlier. For instance, if the predetermined 
time is originally 1/30 second and, however, one frame 
is decoded within 1/60 second, the predetermined time 
allocated for the next frame may be 1/30 + (1/30 - 1/60) 
= 1/20 second. In this case the playback for every 1/30 
second can be realized in time. 
[0016] The simplifying unit may conduct the simplified 
process by automatically setting a high frequency com- 
ponent of the image to zero. In the case of the decoding 
process based on JPEG 2000, the HH sub-band may 
be regarded as invalid coefficients or all zeros. 
[0017] The simplifying unit may adjust the resolution 
of a decoded image, which has a low frequency compo- 
nent as a basic component after the high frequency 
component is automatically set to zero, when the de- 
coded image does not have a certain predetermined lev- 
el of resolution. In the case of decoding based on JPEG 
2000, if only the LL sub-band is decoded within the pre- 
determined time, the image size or the resolution is just 
one-fourth the size of the decoded image to be obtained 
finally (hereinafter this final decoded image is simply re- 



ferred to as "the decoded image"). Therefore the other 
sub-bands may be regarded as zero or this LL sub-band 
image may be scaled up twice in x, y directions and out- 
put. 

5 [0018] The simplifying unit may skip a process of that 
part of the image which corresponds to a low-order bit- 
plane of a low frequency component whenever the low 
frequency component to be decoded in an earlier de- 
coding step is not completely decoded within the prede- 
temnined time. Namely, in the case of JPEG 2000, even 
when all parts of the LL sub-band cannot be decoded, 
the low-order bit-planes of the LL sub-band may be ig- 
nored. 

[0019] The simplifying unit may conduct a normal de- 
coding process on a frame which is being decoded at 
the moment when the playback of the motion picture, 
which is the purpose of the decoding, is stopped, and 
the simplified process is discontinued. If a user directs 
a stop or pause of the playback, the last frame to be 
displayed may be given plenty of decoding time. 
[0020] According to still another embodiment, an im- 
age decoding apparatus is provided. The apparatus 
comprises a shooting block, a block which controls the 
shooting block mechanically, and a processing block 
which processes a digital image obtained by the shoot- 
ing block. The processing block comprises a decoding 
unit which decodes a coded image generated from the 
digital image, and a simplifying unit which switches an 
ensuing part of a decoding process on any frame includ- 
ed in the coded image to a simplified process whenever 
the said frame is not completely decoded within a pre- 
determined time. 

[0021] According to still another embodiment, an im- 
age decoding apparatus is provided. The apparatus 
comprises a receiving block which receives a signal, a 
processing block which processes the received signal, 
and a playback block which plays back the processed 
signal. The processing block comprises a decoding unit 
which decodes a coded image which comprises a digital 
Image extracted from the received signal, and a simpli- 
fying unit which switches an ensuing part of the decod- 
ing process on any frame included in the coded image 
to a simplified process whenever the said frame is not 
completely decoded within a predetermined time. 
[0022] Moreover, any arbitrary replacement or substi- 
tution of the above-described structural components 
and the steps, expressions replaced or substituted in 
part or whole between a method and an apparatus as 
well as addition thereof, and expressions changed to a 
system, a computer program, a data structure, record- 
ing medium or the like are all effective as and are en- 
compassed by the present Invention. 
[0023] This summary of the invention does not nec- 
essarily describe all necessary features, so that the in- 
vention may also be a sub-combination of these de- 
scribed features. 

[0024] The invention will be now described by way of 
example with reference to the accompanying drawings, 
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throughout which like parts are referred to by like refer- 
ences, and in which: 

Fig. 1 shows a procedure of JPEG 2000 by which 
coded ima ge data are decoded. 
Fig. 2 is a block diagram of an image decoding ap- 
paratus according to one embodiment. 
Fig. 3 shows the relationship between a frame pe- 
riod, a time limit and an actual decoding process in 
the embodiment. 

Figs. 4a, 4b and 4c show a simplified process con- 
ducted by a simplifying unit. 
Figs. 5a and 5b show the relationship between an 
LL sub-band and bit-planes In a layer in JPEG 2000. 
Fig. 6 shows how the time limits of other decoding 
steps are rescheduled when a temporal space is 
produced in one deco ding step in the embodiment. 
Fig. 7 is a block diagram of a digital camera accord- 
ing to another embodiment. 
Fig. 8 is a block diagram of a television receiver ac- 
cording to yet another embodiment. 

[0025] The invention will now be described by refer- 
ence to the preferred embodiments. This does not in- 
tend to limit the scope of the present invention, but to 
exemplify the invention. The below embodiments relate 
to apparatus for decoding image data coded by JPEG 
2000. 

[0026] Fig. 1 illustrates a decoding process based on 
JPEG 2000. As shown in the figure, a coded image CI 
is first Input and processed by an arithmetic decoding, 
bit-plane decoding and so forth as described later and 
then processed by an inverse quantization. In this stage, 
an image, which is generated after a wavelet transform 
is conducted twice on an original image, hereinafter re- 
ferred to as the second layer image WI2, is obtained. 
Thereafter this Image is processed by a wavelet Inverse 
transform and the first layer image WI1 is generated. 
This image is then processed again by the wavelet in- 
verse transform and then a decoded Image Dl is ob- 
tained. 

[0027] Now a coding procedure is presented for ease 
of understanding. The coding can be regarded as an in- 
verse transform of the process of Fig. 1 . Namely, the 
decoded image Dl of Fig. 1 corresponds to an original 
image and a wavelet transform is conducted once and 
thereby the first layer image WI1 is generated. The filter 
of the wavelet transform employed in JPEG 2000 is a 
Daubechies filter and the essence is that the filter simul- 
taneously worths as a low-pass filter and a high-pass fil- 
ter in X, y directions of the image. Therefore it divides 
the original image into four frequency sub-bands: the LL 
sub-band which is composed of low frequency compo- 
nents in X and y directions, the HL and LH sub-bands 
which are composed of low frequency components in 
one of the x and y directions and high frequency com- 
ponents in the other direction, the HH sub-band which 
is composed of high frequency components in x and y 
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directions. This filter also has a function of reducing the 
number of pixels comprising the image to half in x, y di- 
rections respectively Thus the four sub-band images 
are generated, as illustrated in Fig. 1 (herein denoted 
5 byLL1,HL1,LH1,andHH1). 

[0028] In the wavelet transform for coding, the filtering 
is conducted a predetermined number of times. In the 
case of Fig. 1 , the wavelet transfomi is conducted twice 
and the second layer image W12 is generated. The sec- 
ond wavelet transform and any subsequent wavelet 
transforms are conducted on the LL sub-band compo- 
nent of the previous layer image. Thus the LL1 sub-band 
of the first layer image Wil is divided into four sub-bands: 
LL2, HL2, LH2, and HH2. In the coding process, the im- 
age is processed by a quantization and others and then 
finally the coded image data CI are obtained. 
[0029] It is to be noted that a low frequency compo- 
nent of the original image appears at the upper left in 
the hierarchized image. In the case of Fig. 1, the LL2 
sub-band at the upper left corner of the second layer 
image WI2 comprises the lowest frequency compo- 
nents, and in other words, with this LL2 sub-band the 
basic nature of the original image can be reconstructed. 
This factor is employed in the following embodiments. 
[0030] Fig. 2 shows the configuration of an image de- 
coding apparatus 1 0. The image decoding apparatus 1 0 
may be a normal computer and comprises a CPU, mem- 
ory and program modules to decode images loaded in 
the memory. The blocks in Fig. 2 depict functions char- 
acteristic of the present embodiment and those skilled 
In the art understand the functional blocks can be em- 
bodied as hardware only, software only or any combi- 
nation of the two. 

[0031] The image decoding apparatus 10 mainly 
comprises a decoding unit 12 and a simplifying unit 30. 
The decoding unit 12 comprises a stream analyzer 14 
which receives coded image data CI and analyzes the 
data stream, an arithmetic decoder 16 which conducts 
an arithmetic decoding on the sequence of the data 
which turned out to be decoded as a result of the anal- 
ysis, a bit-plane decoder 18 which decodes each color 
component of the data obtained thereby in a bit-plane 
form, and an inverse quantizer 20 which conducts an 
inverse quantization on the resultant data, and a wavelet 
inverse transformer 24 which conducts a wavelet in- 
verse transfonm on the n-th layer image Win obtained 
by means of the inverse quantization. The wavelet in- 
verse transformer 24 uses a frame buffer 22 as a work 
area. A decoded image Dl obtained when the decoding 
is finally completed is output from the frame buffer 22 
for any required purpose. 

[0032] In addition, the simplifying unit 30 includes an 
automatic transformer 32 which monitors the decoding 
process and automatically switches a later part of the 
decoding process to a simplified process when the 
elapsed time exceeds a predetermined time limit, a time 
limit table 34 which stores a time limit to be referred to 
by the automatic transfomner 32, and a playback stop 
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detector 36 which detects whether a user directs to stop 
the playback of a motion picture. When the user indi- 
cates pause or stop during the playback of a motion pic- 
ture, the decoded and played frame is actually at that 
moment released from the decoding time limit function, s 
Therefore, when the playback stop detector 36 detects 
such an indication, the simplified process by the auto- 
matic transfonmer 32 is not activated and a normal de- 
coding process and playback are conducted. It is to be 
noted that the automatic transfomner 32 may have al- 
ready initiated the simplified process and in such a case 
the remaining part of the decoding process may be con- 
ducted as normally as possible. 
[0033] The automatic transformer 32 refers to a clock 
CLK to measure the elapsed time. The frequency of the 
clock CLK is divided inside the automatic transformer 
32 and measured by means of a counter which is not 
shown in the figure so that a predetermined time is 
measured. It is to be noted that the automatic transform- 
er 32 may refer to an extemal timing mechanism such 
as Programmable Interrupt Timer (PIT) and so forth, in- 
stead of using the clock CLK. Although the automatic 
transformer 32 herein monitors the progress of the proc- 
ess by the wavelet inverse transfomner 24, it may mon- 
itor the elapsed time of the process in any stage from 
the stream analyzer 14 toward the wavelet inverse 
transform 24. In the following, for the sake of simplicity, 
such possible processes to be monitored are represent- 
ed by the term "decoding process". 
[0034] Fig. 3 shows the relationship between the time 
limit and the elapsed time at each step of the decoding 
process, and the simplified process to be initiated. As in 
the figure, a frame period Tf is set to 1/24 or 1/30 second, 
for example, in the case of decoding a motion picture. 
Three time limits are set, depending on the process 
steps. The decoding of JPEG 2000 processes lumi- 
nance Y and chrominance Cb, Cr independently, which 
are color components. In the figure, the time limits on 
decoding are defined as Ty for the luminance Y, Tb for 
the chrominance Cb, and Tr for the chrominance Cr. 
These time limits are stored beforehand in the time limit 
table 34. Defining the time limit for each of the color com- 
ponents prevents an unnatural image from being re- 
stored, for instance, one in which only one color com- 
ponent is decoded and In which the other color compo- 
nents are not decoded at all. The ratio of the time limits 
may be set on the basis of the ratio of the average times 
necessary for each of the process steps. A larger value 
of the time limit may be set for an important color com- 
ponent, for instance the luminance Y. Any value can be 
set on the time limit, according to experimental results. 
[0035] In JPEG 2000, the decoding process is con- 
ducted in the sequence order of the LL, HL or LH, and 
HH sub-bands of the same layer image. Therefore, four 
sub-bands are first decoded in this order in respect of 
the luminance Y, and these four sub-bands are then de- 
coded in respect of the chrominance Cb and Cr respec- 
tively. 
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[0036] If the processing of the three sub-bands LL, 
HL, LH is nomnally completed in the process of decoding 
the luminance Y, but the elapsed time nevertheless ex- 
ceeds the time limit Ty during the processing of the last 
sub-band HH: in this case, the decoding process for the 
luminance Y is interaipted and switched to the simplified 
process denoted by "process A". Next, the decoding 
process for the chrominance Cb starts. If the decoding 
process of the LL sub-band is completed in the normal 
way, however, the elapsed time exceeds the time limit 
Tb during the process of decoding the following HL sub- 
band: in this case, the decoding process for the chromi- 
nance Cb is interrupted and switched to the simplified 
process denoted by "process B". Then the decoding 
process for the chrominance Cr starts. If the elapsed 
time exceeds the time limit Tr during the processing of 
the first sub-band LL in this decoding process: in this 
case, the decoding process is interrupted and switched 
to the simplified process denoted by "process C". The 
automatic transformer 32 compares the actual elapsed 
time with the time limit for each of the decoding proc- 
esses and initiates the processes A, B and C, as appro- 
priate. 

[0037] Figs. 4a, 4b and 4c illustrate respectively the 
processes A, B and C conducted by the automatic trans- 
former 32. As shown in Fig. 4a, the automatic transform- 
er 32 invalidates the coefficients of the HH sub-band, 
the processing of which is thus canceled during the de- 
coding. Namely, all the components of the HH sub-band 
are replaced by zero. The automatic transformer 32 
works in cooperation with the wavelet inverse transform- 
er 24 for this purpose so that the automatic transformer 
32 can store "zero" into the region where the wavelet 
inverse transformer 24 would originally have stored the 
components of the HH sub-band after the inverse trans- 
form. By means of this simplified process, the process 
of decoding the luminance Y can be conducted within 
the time limit Ty and the degradation of Image quality 
can be minimized by exploiting the fact that the human 
eye is not very sensitive to high frequency components. 
[0038] Fig. 4b shows the simplified process conduct- 
ed as the process B by the automatic transformer 32. 
As in the figure, only the sub-band LL is normally de- 
coded in the decoding process for the chrominance Cb. 
For this reason, all the components of the other three 
sub-bands are invalidated and replaced with zero as 
shown in the figure. In this case the purpose is also to 
preserve the low frequency components. In general, the 
filter of the wavelet transform adopted in JPEG 2000 is 
designed to maintain the image quality in a low bit rate 
so that an image of relatively high quality can be ob- 
tained in the playback of the image by using the LL sub- 
band only. This characteristic is herein utilized. 
[0039] Fig. 4c shows the simplified process C as con- 
ducted by the automatic transfomier 32. In the process 
of the chrominance Cr. not even the first sub-band LL is 
completed. For this reason, as denoted by LL' in the fig- 
ure, a process is conducted on the data of the LL sub- 
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band. In more concrete terms, the processing of the low- 
order bit-planes of a plurality of bit-planes, which com- 
pose the LL sub-band, is skipped so that the processing 
time can be shortened. 

[0040] Fig. 5a shows the relationship between the 
sub-band LL2 in the second layer WI2 and the bit- 
planes. As shown in the figure, in the second layer im- 
age WI2, the sub-band LL2 is first decoded across all 
the bit-planes, as denoted by the rectangular solid 50. 
Therefore if the bit-planes close to the least significance 
bit (LSB) are skipped, the processing time can be short- 
ened while degradation of the image quality is mini- 
mized. 

[0041] As shown in Fig. 5b, the rectangular solid 50 
is divided into the part 52 that is composed of the valid 
bit-planes and the part 54 to be skipped. Here the lowest 
bit-plane is discarded in relation to the time limit Jr. 
[0042] As described above, owing to the simplified 
process performed by the simplifying unit 30, especially 
the automatic transformer 32, one frame can be re- 
stored at each frame period Tf and a motion picture or 
images continuously shot can be played back without 
frame drop. At the same time, a relatively natural image 
can be obtained because degradation of the image qual- 
ity is carefully prevented. Furthermore, since a time limit 
is allocated for each of the color components, it is un- 
likely that color imbalance between the frames will oc- 
cur. Therefore, according to the present embodiment, in 
practice, very large advantages can be produced with a 
relatively small-scale configuration. 
[0043] The case in which the elapsed time exceeds 
the time limit during the process of decoding the image 
has been considered above, however, by way of con- 
trast, the time limit may be left on, depending on condi- 
tions. Fig. 6 shows how the time limit is rescheduled in 
such a situation. 

[0044] In the figure, the frame period Tf is the same 
as that of Fig. 3 and the three time limits Ty, Tb and Tr 
are also the same as those of Fig.3. If the decoding proc- 
ess for the luminance Y starts and alt of the four sub- 
bands LL, HL, LH, HH are completed within the time limit 
Ty: in this case, a temporal space Tm for the time limit 
Ty is left as shown in the figure. This temporal space Tm 
can be used in the decoding processes for the chromi- 
nance Cb and Cr. Therefore, as shown In the figure, the 
time limits Tb and Tr for the chrominance Cb and Cr are 
relaxed and changed to the longer time limits Tb* and 
Tr* respectively. 

[0045] According to this method, if the decoding proc- 
ess for one color component is completed, a temporal 
space can be produced and used for the decoding proc- 
ess of other color components as an effect, resulting in 
an increase in the number of the color components 
which are completely decoded. Therefore image quality 
can be enhanced while color balance is maintained. 
[0046] If all the three color components or any one of 
the components are decoded within the time limit, the 
whole decoding time for this frame might be shorter than 



the originally defined frame period Tf. In this case a sim- 
ilar rescheduling method can be applied between the 
frames. Namely if there is any temporal space when one 
frame is decoded, this temporal space can be added to 

5 the next frame period Tf . According to this method , more 
frames can be restored completely or with a higher im- 
age quality. Moreover, according to this embodiment, a 
desired image can be played at low cost and with low 
power consumption without any excessive load on the 

10 image decoding apparatus 10. 

[0047] Fig, 7 shows the configuration of a digital cam- 
era 200 according to another embodiment. The digital 
camera 200 comprises a shooting block 202, a mecha- 
nism control block 204, a processing block 206, an LCD 

IS monitor 208 and operation buttons 210. 

[0048] The shooting block 202 comprises a lens, a 
lens stop, an optical low-pass filter (LPF), a CCD and a 
signal processor (not shown). The CCD accumulates 
charge according to the amount of light received from 

20 an object imaged on the light receiving plane of the 
CCD. The charge is read as a voltage signal, which is 
decomposed into R, G and B components in the signal 
processor. The components are subjected to white bal- 
ance adjustment and gamma correction. The R, G and 

25 B signals are then input into an A/D converter from which 
digital image data are output to the processing block 
206. 

[0049] The mechanism control block 204 controls the 
optical system including the zooming, focusing and iris 

30 of the shooting block 202. The processing block 206 
comprises a CPU 220, memory 222. a YC processor 
226, a card controller 228 and a communication unit 
224. The image decoding apparatus 10 of Fig. 2 is re- 
alized with some of the functions of the CPU 220, and 

35 an image decoding program loaded in the memory 222. 
The frame buffer 22 of Fig. 2 is realized using part of the 
memory 222. It is to be noted that an image coding ap- 
paratus not shown in the figure is also realized using the 
CPU 220 and the memory 222 so that this digital camera 

40 200 can save image data into a memory card 230. 
Therefore the configuration by which both image coding 
and decoding can be conducted is explained below. 
[0050] From the digital image data the YC processor 
226 generates luminance Y. chrominance Cb and Cr. 

45 The luminance and chrominance are coded separately 
and sequentially. The coded image data CI are output 
via the communication unit 224 or are written into the 
memory card 230 via the card controller 228. 
[0051] The communication unit 224 conducts a pro- 

so tocol transform in accordance with a standardized com- 
munication specification. The communication unit 224 
may transmit data from and to a printer, a game player 
and other extemal devices via respective interfaces. 
[0052] The LCD monitor 208 displays a motion pic- 

55 ture. images from a high-speed continuous shoot and 
still images as well as shooting/replay mode, zoom ratio, 
date and so on. Therefore, when a user shoots a motion 
picture, the picture is first compressed and recorded into 



6 



11 



EP 1 233 625 A2 



12 



the memory card 230, for Instance. When the user plays 
back the motion picture, the simplified process charac- 
teristic of the embodiment is conducted if necessary. 
The operation buttons 210 may include a power switch, 
a release switch and so on to shoot a picture or to set 
operational modes. 

[0053] The above-mentioned configuration has the 
following effects. 

1 . When a still image Is shot and played back, there 
is technically no time limit set on the decoding proc- 
ess. Therefore the decoding is not simplified and 
the image is restored with the highest image quality. 
However, since too long a decoding time might give 
rise to problems with even a still image, the simpli- 
fied process characteristic of this embodiment may 
be conducted in this case. 

2. When a motion picture is shot and played back, 
the simplified process Is conducted depending on 
conditions, because its frame rate must be main- 
tained. Therefore, even if the image decoding ap- 
paratus 1 0 is not highly speeded up, the image qual- 
ity can be maintained and frame drop can be pre- 
vented. Since a high specification is not required for 
the image decoding apparatus 10, advantages in 
cost and power consumption can be gained. 

3. When stop or pause is indicated during the play- 
back of a motion picture, the simplified process is 
not conducted on the last frame and its image qual- 
ity becomes the highest or close to the highest pos- 
sible. In particular, since a user will look at this frame 
as virtually a still image for a relatively long time, 
there may be great advantage in this function. 

[0054] Fig. 8 shows the configuration of a television 
receiver 300 according to yet another embodiment. The 
television receiver 300 includes an antenna 302, a re- 
ceiving block 304 which receives a broadcast wave via 
the antenna 302, a processing block 306 which proc- 
esses image and audio data obtained through the proc- 
ess applied by the receiving block 304, and a play-back 
block 308 which plays the audio and image data decod- 
ed by the processing block 306. An interface block 336 
outputs the image data decoded by the processing block 
306 to an appropriate extemal device. 
[0055] The receiving block 304 Includes a tuner 320 
and a packet separator 322. The tuner 320 selects a 
transponder which includes a channel selected by a us- 
er and conducts a QPSK demodulation. A stream which 
Includes a plurality of transport packets obtained In the 
demodulation is forwarded to the packet separator 322. 
The packet separator 322 is a demultiplexer and it sep- 
arates a packet corresponding to a desired channel and 
outputs this to the processing block 306. 
[0056] An image and audio decoder 334 in the 
processing block 306 wori^s in cooperation with a CPU 
330 and a memory 332 and thus decodes the Image and 
audio data which are coded and transmitted by a broad- 



casting station. The image and audio decoder 334 de- 
codes an input packet and outputs audio data to an au- 
dio output unit 340 and image data to a display appara- 
tus 344. The audio output unit 340 conducts a predeter- 

5 mined process on the input audio data and finally audio 
is output to a speaker 342. That part of the configuration 
of the processing block 306, that comprises the image 
and audio decoder 334, the CPU 330, and the memory 
332, which relates to the image decoding, corresponds 

10 to the image decoding apparatus 10 of Rg. 2. The 
above-mentioned configuration can realize a so-called 
digital TV set at very low cost and with low power con- 
sumption. This TV set can be incorporated into a small- 
sized device such as a mobile phone. 

IS [0057] Although the present invention has been de- 
scribed in the embodiments, many changes, modifica- 
tions and replacements may be made by those skilled 
In the art without departing from the scope of the present 
Invention, which is defined only by the appended claims. 

20 Such changes and substitutions may be provided as fol- 
lows. 

[0058] The process B of Fig. 4b invalidates all the 
components of the three sub-bands except the LL sub- 
band. However, as another method, the components 

25 other than the LL sub-band may be left indefinite and 
the LL sub-band may simply be scaled up twice in x, y 
directions. In this case an interpolation process between 
pixels may be conducted, however, if the filter used for 
this purpose has the same characteristics as that used 

30 In the wavelet inverse transfomi of JPEG 2000, the 
same image can be obtained as in the case in which the 
components of the three sub-bands are invalidated. 
[0059] In the embodiments, a time limit is allocated for 
each component. The time limit, however, may be allo- 

35 cated according to other Image attributes. For instance, 
if the image is divided into a plurality of regions and the 
decoding process is conducted on each of the regions, 
a time limit may be allocated for each of the different 
regions. In this case, the time limit may be allocated in 

40 proportion to the dimension of the region. A relatively 
long time may be allocated to a preeminent region of the 
image, for instance, a region with high luminance. A rel- 
atively long time may also be allocated to a region with 
a large amount of movement, for instance, a region 

45 which has large motion vector components. A longer 
time may be allocated to a region of high importance as 
an object, for instance, the region of a human face on a 
TV phone. Such allocation may be considered in various 
ways. Moreover, it is possible to divide the image into 

50 tiles beforehand In JPEG 2000 and in this case the time 
limit may be allocated to each of the tiles in a similar 
manner. In this case, if the process of one region is com- 
pleted within the time limit, the time limit of the other re- 
gions may consequently be rescheduled likewise. 

55 [0060] In Fig. 4c the processing of the low-order bit- 
planes of the LL sub-band is omitted. As another meth- 
od, the LL sub-band of a previously decoded layer may 
be utilized. For instance, if the decoding process for the 
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ensuing part of the decoding process of any frame 
included in the coded image is switched to the sim- 
plified process whenever the frame is not complete- 
ly decoded within the predetermined time. 

5 

3. The method of Claim 2. further comprising dividing 
the decoding process of the frame into a plurality of 
steps and defining a time limit for each of the steps, 
and wherein when an elapsed time reaches the de- 

10 fined time limit at any one of the steps, the ensuing 
part of the processing of the step is simplified. 

4. The method of any one of Claims 2-3, wherein the 
coded image is a motion picture in which the frames 

15 are coded independently of each other and the sim- 
plified process is a process confined within each of 
the frames. 

5. An image decoding apparatus comprising: 

20 

a decoding unit (1 2) which decodes a coded im- 
age; and 

a simplifying unit (30) which switches an ensu- 
ing part of a decoding process to a simplified 
25 process whenever the coded image is not com- 

pletely decoded within a predetermined time. 

6. The apparatus of Claim 5, wherein the coded image 
is a motion picture coded on a frame basis; and the 

30 simplifying unit (30) switches the ensuing part of the 
decoding process on any frame included in the cod- 
ed image to the simplified process whenever the 
frame is not completely decoded within the prede- 
termined time. 

35 

7. The apparatus of Claim 6, wherein the simplifying 
unit (30) compares a time limit defined for each of 
a plurality of frame-decoding steps to an elapsed 
time at each of the steps, and simplifies an ensuing 

40 process of any one of the steps in which the elapsed 
time reaches the time limit. 

8. The apparatus of Claim 7, wherein the simplifying 
unit (30) conducts the comparison of the time limit 

45 and the elapsed time and activates the simplified 
process if necessary at the step of decoding each 
of a plurality of color components which compose 
the image. 

50 9. The apparatus of Claim 7. wherein the simplifying 
unit (30) conducts the comparison of the time limit 
and the elapsed time and activates the simplified 
process if necessary at the step of decoding each 
of a plurality of regions which compose the image. 

55 



sub-band LL1 of the first layer image WI1 of Fig. 1 is not 
completed, the sub-band LL2 of the second layer image 
WI2, which is a previous layer, may be enlarged and uti- 
lized. 

[0061] In Fig. 4c, the processing of the low-order bit- 
planes of the LL sub-band is omitted so that the process- 
ing time can be shortened. As another decoding meth- 
od, the process of four sub-bands may be conducted on 
a bit-plane basis starting from the highest bit-plane in 
order of the bit-plane layers. Namely, after the process 
is conducted in the sequence order of the LL, HL or LH, 
and HH sub-bands on one bit-plane, the process is con- 
ducted in the same order on another lower-order bit- 
plane. In this decoding method, if the time limit is ex- 
ceeded, the processing of the low-order bit-planes that 
are not yet processed may be omitted and thereby the 
decoding process can be simplified. Unlike the case 
shown in Figs. 5a and 5b, the processing of the low- 
order bit-planes is omitted on the whole four sub-bands 
in this case. 

[0062] Further inventive features of the present em- 
bodiments are defined in the following paragraphs: 

1. An image decoding apparatus comprising a 
shooting block, a block which controls the shooting 
block mechanically, and a processing block which 
processes a digital image obtained by the shooting 
block, wherein the processing block comprises a 
decoding unit which decodes a coded image gen- 
erated from the digital image, and a simplifying unit 
which switches an ensuing part of a decoding proc- 
ess on any frame included in the coded image to a 
simplified process whenever the frame is not com- 
pletely decoded within a predetermined time. 

2. An image decoding apparatus comprising a re- 
ceiving block which receives a signal, a processing 
block which processes the received signal, and a 
playback block which plays back the processed sig- 
nal, wherein the processing block comprises a de- 
coding unit which decodes a coded image which 
comprises a digital image extracted from the re- 
ceived signal, and a simplifying unit which switches 
an ensuing part of a decoding process on any frame 
included in the coded image to a simplified process 
whenever the frame is not completely decoded 
within a predetermined time. 



Claims 

1. A method for decoding a coded image, comprising 
switching an ensuing part of a decoding process to 
a simplified process whenever the coded image is 
not completely decoded within a predetermined 
time. 

2. The method of Claim 1. wherein the coded image 
is a motion picture coded on a frame basis, and the 



10. The apparatus of any one of Claims 7-9, wherein 
the simplifying unit (30) relaxes the time limit allo- 
cated to a remaining step when any one of the steps 
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is completed within its time limit. 

11. The apparatus of any one of Claims 6-10, wherein 
the simplifying unit (30) relaxes the predetermined 
time allocated to the next frame to be decoded when ^ 
the decoding process of any one of the frames is 
completed earlier. 

12. The apparatus of any one of Claims 6-9, wherein 

the simplifying unit (30) conducts the simplified io 
process by automatically setting a high frequency 
component of the image to zero. 

13. The apparatus of Claim 12. wherein the simplifying 
unit (30) adjusts a resolution of a decoded image, is 
which has a low frequency component as a basic 
component after the high frequency component is 
automatically set to zero, when the decoded image 
does not have a desired level of the resolution. 

20 

14. The apparatus of any one of Claims 6-13, wherein 
the simplifying unit (30) skips a processing of a part 
of the image which corresponds to a low-order bit- 
plane of a low frequency component whenever the 
low frequency component to be decoded In an ear- 25 
Her decoding step is not completely decoded within 

the predetermined time. 

15. The apparatus of any one of Claims 6-13, wherein 

the simplifying unit (30) conducts the simplified 30 
process by skipping a processing of a low-order bit- 
plane of the image. 

16. The apparatus of any one of Claims 6-15, wherein 

the simplifying unit (30) conducts a normal decod- 35 
ing process on a frame which is being decoded at 
a moment when a playback of the motion picture is 
stopped, instead of continuing the simplified proc- 
ess. 

40 

17. The apparatus of any one of Claims 6-16, wherein 
the coded image is a motion picture in which the 
frames are coded independently of each other and 
the simplifying unit (30) conducts the simplified 
process confined within each of the frames. 
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